Skip to content

Add timeouts and bounded reconnect to Redis client#120

Open
masnwilliams wants to merge 3 commits into
mainfrom
hypeship/redis-timeouts
Open

Add timeouts and bounded reconnect to Redis client#120
masnwilliams wants to merge 3 commits into
mainfrom
hypeship/redis-timeouts

Conversation

@masnwilliams

@masnwilliams masnwilliams commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

The Redis client retried reconnecting forever with no connect or command timeout. When Redis was unreachable, any command (e.g. the OAuth org-context writes in /token) blocked until the serverless function limit instead of throwing — so kernel login hung after the browser showed "authentication successful" rather than failing with an error.

This bounds the failure:

  • Bounded reconnectreconnectStrategy now returns an Error after MAX_RECONNECT_ATTEMPTS (10) instead of retrying indefinitely, so queued commands reject when Redis stays down.
  • Connect timeoutconnectTimeout (5s) on the socket so a black-holed connection fails fast.
  • Per-command timeout — every op runs through withTimeout (5s ceiling) so a stalled-but-connected command surfaces as an error.

Net effect: a Redis outage returns a clean error in seconds instead of hanging the caller.

Test plan

  • tsc --noEmit passes
  • prettier clean
  • Manual: confirm normal OAuth login still succeeds against a healthy Redis (behavior unchanged on the happy path)

Follow-up companion PR on kernel/cli adds a timeout on the CLI token exchange so it also fails fast.


Note

Medium Risk
Touches shared Redis infrastructure used by OAuth token and authorize flows; mis-tuned timeouts could cause false failures on slow networks, though happy-path behavior is intended to stay the same.

Overview
When Redis is unreachable, OAuth-related callers (e.g. org-context writes on /token) no longer block until the serverless limit; they fail with an error within a few seconds.

Bounded reconnectreconnectStrategy stops after 10 attempts and returns an Error instead of retrying indefinitely.

Connect path — 5s connectTimeout on the socket plus connectWithTimeout() racing client.connect() against the same ceiling; on timeout it resets state with client.destroy() so the next call does not inherit a long reconnect loop.

Commands — All Redis ops in withReconnect run through withTimeout (5s), including the retry after transient socket errors.

Reviewed by Cursor Bugbot for commit fe21c00. Bugbot is set up for automated code reviews on this repo. Configure here.

Previously the Redis client retried reconnecting forever with no connect or
command timeout, so an unreachable Redis made callers (e.g. the OAuth token
exchange) block indefinitely instead of failing. Bound the reconnect attempts,
set a connect timeout, and cap each command so Redis outages surface as errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
mcp Ready Ready Preview, Comment Jul 20, 2026 4:07pm

@masnwilliams
masnwilliams marked this pull request as ready for review July 13, 2026 20:45
@masnwilliams
masnwilliams requested a review from sjmiller609 July 13, 2026 20:46
Comment thread src/lib/redis.ts
Comment thread src/lib/redis.ts
Comment thread src/lib/redis.ts

@vercel vercel Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

The 5s command timeout does not cover ensureConnected()/client.connect(), so an unreachable Redis blocks the caller for ~60s+ instead of the intended ~5s budget.

Fix on Vercel

sjmiller609
sjmiller609 previously approved these changes Jul 20, 2026
Cap ensureConnected() with the same ~5s ceiling as commands (connect drives
the reconnect loop, so an unreachable Redis blocked callers for the full
reconnect budget), destroy the client on connect timeout so the next call
starts clean, swallow late command settlements to avoid unhandled rejections,
and fix the reconnect-attempt off-by-one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit fe21c00. Configure here.

Comment thread src/lib/redis.ts
});
connectPromise = connectWithTimeout().finally(() => {
connectPromise = null;
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parallel connect destroys shared client

Medium Severity

When a Redis connection attempt fails or times out, connectWithTimeout calls client.destroy() on the shared singleton client. This prematurely terminates other in-flight Redis operations or concurrent connection attempts, breaking Redis for those callers.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit fe21c00. Configure here.

Comment thread src/lib/redis.ts

// connect() drives the reconnect loop, so against an unreachable Redis it blocks
// for the whole reconnect budget. Cap the wait so callers fail within the same
// ceiling as a command instead of after every reconnect attempt.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Late connect after timeout wins

Medium Severity

When the connect deadline wins Promise.race, the code calls client.destroy() but only attaches .catch(() => {}) to the in-flight connect() promise. A late successful connect can still run library side effects, and the global ready listener sets isConnected true, so ensureConnected may skip reconnect while the client was just destroyed.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit fe21c00. Configure here.

Comment thread src/lib/redis.ts
} finally {
if (timer) clearTimeout(timer);
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale state after command timeout

Medium Severity

withTimeout rejects when a command exceeds the deadline but does not clear isConnected or tear down the socket, unlike connectWithTimeout. Later calls can hit ensureConnected's client.isOpen && isConnected early return on a stalled link, so requests keep timing out without attempting a fresh connect.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit fe21c00. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants